Machine learning‐based prediction of clinical outcomes after traumatic brain injury: Hidden information of early physiological time series

Abstract Aims To assess the predictive value of early‐stage physiological time‐series (PTS) data and non‐interrogative electronic health record (EHR) signals, collected within 24 h of ICU admission, for traumatic brain injury (TBI) patient outcomes. Methods Using data from TBI patients in the multi‐center eICU database, we focused on in‐hospital mortality, neurological status based on the Glasgow Coma Score (mGCS) motor subscore at discharge, and prolonged ICU stay (PLOS). Three machine learning (ML) models were developed, utilizing EHR features, PTS signals collected 24 h after ICU admission, and their combination. External validation was performed using the MIMIC III dataset, and interpretability was enhanced using the Shapley Additive Explanations (SHAP) algorithm. Results The analysis included 1085 TBI patients. Compared to individual models and existing scoring systems, the combination of EHR and PTS features demonstrated comparable or even superior performance in predicting in‐hospital mortality (AUROC = 0.878), neurological outcomes (AUROC = 0.877), and PLOS (AUROC = 0.835). The model's performance was validated in the MIMIC III dataset, and SHAP algorithms identified six key intervention points for EHR features related to prognostic outcomes. Moreover, the EHR results (All AUROC >0.8) were translated into online tools for clinical use. Conclusion Our study highlights the importance of early‐stage PTS signals in predicting TBI patient outcomes. The integration of interpretable algorithms and simplified prediction tools can support treatment decision‐making, contributing to the development of accurate prediction models and timely clinical intervention.


| INTRODUC TI ON
Traumatic brain injury (TBI) encompasses cranial injuries caused by traffic accidents, falls, sports, and war.Globally, TBI accounts for approximately 69 million cases each year, constituting a significant cause of death and disability.About 10% of TBI cases necessitate intensive care unit (ICU) admission. 1,2entifying early predictors of patient prognosis and outcomes, as well as implementing effective treatments, poses major challenges in TBI research. 3The APACHE score is commonly utilized to forecast the severity and prognosis of critically ill patients. 4However, obtaining timely indicators remains a challenging endeavor.For instance, TBI patients admitted to the emergency department often present in a state of coma, impeding the timely collection of detailed information regarding chronic comorbidities. 5Moreover, gathering over 100 variables required for calculating the APACHE score poses additional difficulties due to limited medical resources available.
In recent years, machine learning (ML) has gained significant attention and recognition in the clinical field due to advancements in statistical theory and computer technology.However, most studies only incorporate repeated measurements of the same variable at different time points (longitudinal data).A study of publications from 2009 to 2016 revealed that less than 8% of predictive models included longitudinal data as time-varying covariates. 6Neglecting the potential of these high-dimensional, longitudinal, time-varying covariates to improve prediction, as demonstrated in cardiovascular disease risk prediction, 7,8 is a prevailing issue.
In this study, we propose a novel method for predicting clinical outcomes of TBI patients, including survival, neurological function, and prolonged length of ICU stay (PLOS), based on clinical electronic health record (EHR) data and physiological time-series (PTS) signals.Additionally, we employ interpretable ML techniques to elucidate the best-performing models and gain valuable clinical insights.The findings are validated using comprehensive data from the multi-center Philips eICU-CRD database 9 and additionally validated using the Medical Information Mart for Intensive Care (MIMIC) III database. 10

| Study design and protocol
The study design was implemented as depicted in graphical abstract image.The author of this study has successfully completed the Collaborative Institutional Training Initiative examination (certification number 49437998) and was granted access to both databases.
The author was responsible for extracting the data.Given that the study involved the analysis of anonymized publicly available databases, which had obtained prior institutional review board approval from the Massachusetts Institute of Technology and BIDMC, the requirement for informed consent was waived.

| Study samples
All patients included in this study were sourced from the eICU version v2.0 databases and MIMIC-III version v1.4.Patients with a diagnosis of TBI were identified in both databases using International Classification of Diseases-9 diagnosis codes (Table S6).The study employed the following inclusion criteria: (1) admission to the ICU following TBI; (2) age between 18 and 89 years; (3) ICU stay duration exceeding 24 h; (4) availability of PTS signals; and (5) presence of Glasgow Coma Scale (GCS) records at admission and motor GCS records within 24 h of discharge.In cases where patients had multiple ICU admissions, only data from the initial ICU admission during the first hospitalization were considered for analysis.One point that needs to be mentioned is that PTS signals are not currently available in MIMIC-IV; hence, they cannot be obtained.Therefore, MIMIC-III database was used for external validation, despite it being released earlier.

| Variable extraction
Clinical variables from the initial 24 h of ICU stay for TBI patients were extracted from both databases using structured query language for further analysis.The EHR data collected encompassed various domains, including demographics, comorbidities, laboratory results, vital signs, medical treatments, and scoring systems.An overview of the selected EHR features is provided in Table S7.
PTS data in the eICU database were recorded at 5-min intervals and included heart rate (HR), systolic and diastolic blood pressure (SBP, DBP), respiratory rate, and oxygen saturation measured by pulse oximeter (SpO2).Similarly, PTS data for patients in the MIMIC-III database were extracted from the MIMIC-III waveform database 11 based on matching identification.To achieve the objectives of our study, we specifically focused on the PTS data obtained within the initial 24 h following ICU admission for the aforementioned five signals.Each patient's PTS data were stored individually in a separate file to facilitate subsequent feature extraction analysis.

| Outcome definition
This study aimed to investigate three primary clinical outcomes: in-hospital mortality, neurological status at hospital discharge, and PLOS.Neurological outcome was assessed using the motor subscore of the Glasgow Coma Score (mGCS) recorded at discharge, where a favorable outcome was defined as an mGCS score of 6 and an unfavorable outcome as an mGCS score ≤5.PLOS was considered as a binary variable, categorized based on the 75th percentile length of ICU stay among the study participants.In the eICU cohort, a PLOS of 5 days or longer was classified as prolonged, while in the MIMIC III cohort, a PLOS of 16 days or longer was considered prolonged.

| Study participants
Initially, the entire population from the two databases was screened, resulting in the identification of 4809 ICU admissions after TBI from the eICU-CRD database and 2812 admissions from the MIMIC-III database, respectively.Following the application of exclusion criteria, a total of 1085 patients were ultimately included in the study cohort.
Detailed summaries of the characteristics and outcome distributions of the TBI cohorts are provided in Tables 1 and S8, respectively.S4-S7, along with Tables S8 and S9, present the performance of all predictive models for each clinical outcome.The evaluation encompassed different feature subsets, including EHRonly, PTS-only, and the combined EHR and PTS, as well as various scoring systems such as APACHE, APS, GCS, and SOFA.

| Model performance
From the summary of the testing set results, we obtained the following key findings.Our results demonstrate that the individual EHR data achieved optimal AUC values greater than 0.8 for all three modeling tasks, specifically 0.869, 0.863, and 0.803, respectively.An intriguing finding is that the combination of EHR and PTS data enhances the predictive performance of all ML models, with the highest increase in AUROC being 16.3% compared to using EHR data alone.Among the three ML models, the XGBoost model demonstrates the highest discriminative ability, with AUROCs of 0.878, 0.877, and 0.835 for predicting in-hospital mortality, neurological outcome, and PLOS, respectively.Furthermore, the decision curve analysis (DCA) indicates that the XGBoost model achieves optimal net benefit.
In order to thoroughly assess the predictive performance of our models, we conducted a comprehensive comparison with several existing scoring systems.An interesting observation is that using EHR data alone has already achieved comparable, or even surpassing, performance compared to existing scoring systems for neurological outcome and PLOS.In predicting in-hospital mortality, the APACHE scoring system demonstrates impressive predictive performance with the highest AUROC (0.908).

| Most important predictors of TBI
The SHAP algorithm was conducted to visually exhibit each factor's importance to the three clinical outcomes predicted by the optimal model XGBoost.Figure 1 shows the feature importance plot, including the top 20 selected EHR and PTS variables in descending order.
Briefly, a higher intensity of red indicates a greater risk impact, while a higher intensity of blue indicates a greater protective impact.
In terms of in-hospital mortality and neurological functional status, the use of mechanical ventilation appears to be the most significant factor, while the maximum temperature within 24 h of admission is associated with the state of PLOS.As depicted in Figure 2, we summarize the derived categories of PTS generated by HCTSA.
In terms of all clinical outcomes, the main features contributing to inhospital mortality are HR, SBP, and SPO2; SBP is the primary derived feature for neurological functional status, and DBP is the primary derived feature for PLOS.Further observations reveal that PTS features account for a considerable proportion, unexpectedly reaching 60%, of the top 20 selected EHR and PTS variables for neurological functional status.This reinforces our hypothesis that these PTS features independently contribute to the final model performance.
Furthermore, to obtain the precise forms of EHR factors influencing all clinical outcomes, we provide detailed descriptions using Venn diagrams and SHAP dependency plots for better clinical guidance.The analysis results are shown in Figure 3

TA B L E 2 (Continued)
For in-hospital mortality, high levels of platelet_max, bicarbon-ate_min, creatinine_min, and subdural hematoma, as well as low levels of chloride_min, resp_rat_max, and wbc_max, are identified as individual risk factors (Figure S10).Regarding neurological function, high levels of bun_min and platelet_min, as well as low levels of hemoglobin_max, spo2_min, and the use of vasopressors, are identified as individual risk factors (Figure S11).For PLOS, high levels of hematocrit_min and chloride_max, as well as low levels of bicarbonate_max, are identified as individual risk factors (Figure S12).

| External validation
Compared to eICU, the validation of MIMIC-III demonstrated a decrease in performance: The AUROC of the optimal MIMIC-III model was 0.779 for in-hospital mortality, 0.78 for neurological prognosis, and 0.769 for PLOS, representing a decrease of 9.9%, 9.7%, and 6.6%, respectively.A comprehensive overview of the external validation results is provided in Table S10.

| Online tool for prediction of clinical outcomes
Considering the clinical utility and the fact that EHR data can provide performance superior to or even surpassing traditional scoring systems, we developed an emergency TBI prediction tool (www.tbiproph et.com) based on the optimal predictive model XGBoost and EHR features within 24 h for the three clinical outcomes.This system allows batch prediction and enables result queries based on automatically generated unique IDs.The system interface is illustrated in Figure 4.

| DISCUSS ION
Our research findings demonstrate the value of ML algorithms and routine collection of clinical data within 24 h during intensive care for predicting future clinical events in TBI patients.We observed that the models we constructed exhibited high predictive performance for the three clinical outcomes of interest: survival, neurological status, and PLOS.Such performance is not commonly observed in current clinical prognostic models, and we believe that the inclusion of PTS data and decoding of high-frequency physiological data were key to this improvement.Our proposed approach has several advantages: We utilized a highly specific cohort of patients (i.e., critically ill patients with traumatic TBI) rather than a general population cohort; the data we included were based on routine clinical collection, eliminating the need for additional variables (e.g., the wellknown APACHE model for predicting ICU mortality, which requires Overall plot for the highly comparative time-series analysis physiological time-series-derived features.(A) Simplified distribution heatmap of highly comparative time-series analysis physiological time-series-derived features for the three clinical outcomes (the complete dictionary is provided in Appendix S1).The percentage of physiological time-series data among the top 20 variables in the prediction of outcomes in-hospital mortality (B), neurological status at hospital discharge (C), and prolonged length of ICU stay (D), respectively, in the combined electronic health record and physiological time-series data prediction.collecting 142 variables); and the application of interpretable algorithms can provide insights for monitoring clinical states that change rapidly and aid in treatment decision-making.Importantly, our analysis was conducted using data obtained through routine clinical practice, and we have developed simplified online tools with high predictive performance that can be utilized in clinical settings.This highlights the potential for validation on a larger scale and clinical implementation.
[14] Our research findings further underscore the potential hidden information of collecting PTS data within 24 hours of ICU admission.
Although ML has been extensively developed in the prognosis of TBI patients, PTS has not yet been applied to this type of model. 15,16e development and application of machine learning algorithms offer new opportunities for disease prediction in TBI patients, enabling informed decision-making when sufficient and relevant data are available.We selected KNN, MLP, and XGBoost for our study due to their unique strengths and complementary characteristics.
KNN is one of the simplest machine learning algorithms, which helps in understanding the inherent difficulty of the classification task. 17P, a type of neural network, excels at capturing complex nonlinear interactions in the data, making it well-suited for predicting clinical outcomes where the relationships between variables are not straightforward. 18XGBoost, an ensemble learning method, is known for its robustness against overfitting, providing high predictive accuracy in various clinical applications. 19,20This combination of models allows us to leverage their individual strengths for a comprehensive evaluation of our predictive tasks.
With the rapid development of artificial intelligence technology, the prediction of TBI-related outcomes has entered a new era.
Robson et al. 21utilized machine learning methods to predict early TBI mortality, achieving an AUC of 0.906.However, their model included multiple variables such as GCS scores, pupillary reactivity at admission, and CT scan results, which may not be feasible given the increasingly strained medical resources.Wu et al.This is consistent with our findings, but their top predictive model requires inclusion of 54 clinical variables, significantly more than our study.Shikha et al. 23 explored the value of predicting discharge neurological status based on imaging and clinical data, showing that the integration of multiple data sources achieved the best AUC value of 0.88.This is consistent with a major focus of our study, namely that the comprehensive exploration of diverse data can provide effective prognostic information.In our additional focus on PLOS outcomes, many studies similarly emphasize clinical characteristics associated with PLOS.For example, John et al. 24 identified that the severity of TBI, age, and access to acute post-care are independently associated with prolonged PLOS of ≥28 days, suggesting that high-risk patients should be identified early to provide transitional care resources.This aligns with the aim of our study to provide a simplified automated diagnostic tool incorporating non-human-dependent predictive factors, maximizing predictive efficiency given the current constraints on medical resources and manpower.
In this study, all patients in the eICU-CRD and MIMIC-III databases were screened, and over 1000 TBI patients were adopted as the study subjects.Clinical outcome analysis and validation were conducted through ML technology and HTCSA analysis.One thing we would like to point out is that despite clinical guidelines, there is considerable variation in acute care for TBI patients among different institutions (even within the same ICU). 25We found a decrease in model recognition rates of 6%-10% in external validation in MIMIC III, which is not unexpected.
Given the highly variable and heterogeneous nature of individual clinical characteristics, the clinical interpretation of the features encompassed in risk assessment predictions will be crucial for achieving clinical applicability, necessitating transparency and traceability of the ML decision-making process to be provided to physicians.In our study, we applied the SHAP algorithm to identify several well-known predictive variables, as well as previously unreported variables.An interesting and important finding is that some EHR variables contribute significantly to our predictions for the three clinical outcomes.The clinical consensus for treating TBI suggests that the use of vasopressor agents can maintain appropriate cerebral perfusion pressure, reduce further injury, and improve patient prognosis and outcomes. 26,27However, the use of mechanical ventilation may lead to ventilator-associated pneumonia and potentially impact the patient's clinical progression. 28,29ditionally, temperature control is necessary as abnormal body temperature can also affect the patient's neurological status. 30,31nitoring urine output is crucial as functional impairment and urodynamic abnormalities in TBI patients are associated with adverse outcomes. 32Moreover, elevated blood glucose levels in TBI patients may be related to severity and prognosis, necessitating the monitoring of cerebral glucose levels to predict the possibility of secondary ischemia and reflect treatment efficacy. 33,34rthermore, coagulation dysfunction is a common complication in TBI patients, and maintaining INR within normal levels contributes to improved treatment outcomes and prognosis. 35,36These findings, to some extent, align with the current mainstream reports and indirectly validate the rationale behind the methodology we employed.
It should be noted that, in addition to the variables mentioned above, there may be other influential factors that hold particular significance for individual outcomes.For in-hospital mortality, we identified subdural hematoma and white blood cell count as potential key factors.Similar to our findings, studies have also reported a strong association between subdural hematoma and increased mortality rate. 37Furthermore, elevated white blood cell count has been suggested as an indicator of severe inflammatory response, and thus, monitoring and better management of systemic infections may potentially reduce the mortality rate in TBI patients. 38Regarding the neurological functional status of TBI patients, we observed the critical roles of blood urea nitrogen (BUN) and SpO2.Acute kidney injury is one of the most common and severe complications following TBI. 39Previous research has indicated that intracranial pressure monitoring can guide the use of mannitol to protect renal function. 40Additionally, studies have found that the influence of TBI on mortality rate significantly changes with high oxygen levels. 41However, currently, there is no research exploring the relationship between BUN, SpO2, and the neurological functional status prognosis in TBI patients.PLOS is an important reflection of the severity of TBI.Some of the indicators we identified have received more attention in terms of their impact on TBI mortality, such as chloride, 42 bicarbonate, 43 and hemoglobin. 44The significance of these indicators for PLOS warrants further exploration.
Comprehensive monitoring and management of these key clinical features are crucial for the treatment of TBI patients.However, we want to emphasize that the relationships between the features indicated by our analysis and clinical outcomes are primarily correlational, and the implementation of appropriate intervention measures and confirmation of causal relationships require prospective studies to validate.

| Limitations
Several limitations in this work need to be noted.Firstly, it is based on a retrospective analysis of past data, which may introduce bias and confounding factors, and the presence of unrecorded or missing data may impact the results.Secondly, the study only considered electronic medical record data within the first 24 h of admission and did not account for the influence of subsequent care intensity and life-sustaining treatments during the treatment process.Furthermore, the effectiveness of machine learning algorithms is influenced by the size, quality, and complexity of the dataset and training models, and the decision-making process can be difficult to interpret.Lastly, the predictive capability of machine learning algorithms may be limited when applied to new datasets, primarily due to significant differences in the treatment capabilities of hospitals at different levels, as well as the potential for the

| CON CLUS IONS
In conclusion, our findings demonstrate that ML models trained on EHR and PTS data can successfully differentiate inpatient survival, discharge neurological prognosis, and prolonged length of stay in TBI patients.The high-resolution ICU time-series data also contain valuable hidden information.Furthermore, our models are interpretable and reveal several potentially relevant prognostic features, warranting further exploration.

Figures
Figures S1 and S2 illustrate the feature variable selection results of LASSO regression applied to identify the EHR data and PTS data for each clinical outcome.After conducting a multicollinearity check, it was ensured that the VIF values of the included indicators were all less than 10, as shown in Figure S3.Ultimately, a total of 32 clinical indicators and 8 PTS variables were selected for predicting inhospital mortality.For neurological outcome, 29 clinical indicators and 23 PTS variables were chosen.Lastly, for PLOS, 13 clinical indicators and 11 PTS variables were selected.

F I G U R E 1
Shapley Additive Explanations analysis results of optimal prediction model XGBoost.Rank of importance of the top 20 features for in-hospital mortality (A), neurological status at hospital discharge (B), and prolonged length of ICU stay (C).
22 included 2804 TBI patients to predict in-hospital mortality and found that the XGBoost model could capture information hidden within the demographic and clinical predictive factors of TBI patients, resulting in more accurate predictions compared to the LR method.

F I G U R E 3
The Venn diagram summarizes the common and unique important electronic health record variables of the optimal prediction model, XGBoost, for the three clinical outcomes.F I G U R E 4 Online tools for diagnosing the three clinical outcomes of traumatic brain injury using the first 24 h of electronic health record clinical features.(A) Diagnosis page.(B) Results query page.

| 11 of 12 DING
et al. definition of PLOS to "float" with variations in the target population, necessitating additional cohort validation.Despite these limitations, this study employed SHAP for explanatory analysis and conducted external validation, providing new insights into the analysis and application of electronic medical records for TBI patients and holding potential to guide future clinical practice and decision-making.

Table 2 ,
Figures Demographic and clinical characteristics of the complete datasets.appears to better avoid false negatives and has the potential to improve the identification of high-risk patients.Given the differences in the number of metrics used in actual calculations, we consider this performance decrease acceptable.Furthermore, our optimized model exhibits higher performance in predicting neurological outcome and PLOS compared to the scoring systems.
Although our optimized model experiences a slight decrease of approximately 3% in AUROC, it still outperforms APACHE in terms of sensitivity (0.911 vs. 0.889).Considering the in-hospital mortality rate of 18.2%, our TA B L E 1 Note: Data are n (%) or mean (SD).Abbreviations: APSIII, Acute Physiology Score III; GCS, Glasgow Coma Scale; SOFA, Sequential Organ Failure Assessment.TA B L E 2 Model performance summary for all models for each clinical outcome and feature subset in testing set.model through our external validation, it was demonstrated that the robustness of the XGBoost model and the significance of PTS within it were further confirmed, reinforcing our assertion that PTS data encapsulate a wealth of prognostic information.